Description

Now that we have integration tests from Python packages installed in the .venv directory of software areas, we can proceed to include them in the global bundle script and its helper scripts:

• dunedaq_integtest_bundle.sh
• list_available_integtests.sh
• list_repos_with_integtests.sh

The changes in this Pull Request attempt to do that.

Two notes:

• I have attempted to continue to support the "local" keyword for finding integtests in software packages that are included in a local software area. For Python packages, this corresponds to the presence of the package in the pythoncode subdirectory. (Typically, all Python packages are installed in the .venv subdirectory of a local software area, so the presence of a package in that area can't be used to determine "local" vs. not-local.)
• I have not yet added logic to the bundle script to add the -p no:cacheprovider qualifier to the pytest command when running integtests from Python packages that are not installed in a write-able location on disk. The reason for this is twofold: there are discussions about how to avoid needing to specify that qualifier, so we will let that discussion proceed. And in the short term, I expect that it will be rather rare for someone to run integtests from Python packages without having a local software area.

Here are suggested steps for testing these changes:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_260701_A9 ${DATE_PREFIX}FDDevBundleScriptPythonPkgs_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevBundleScriptPythonPkgs_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b kbiery/python_package_integtests
cd ..

. ./env.sh
dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh -r all --list

echo ""
echo -e "\U1F535 \U2705 Note that the drunc integtest is included in the list of *all* tests which would be run. \U2705 \U1F535"
echo ""
echo ""
sleep 3

dunedaq_integtest_bundle.sh -r local --list

echo ""
echo -e "\U1F535 \U2705 Note that the drunc integtest is *not* included in the list of *local* tests which would be run. \U2705 \U1F535"
echo ""
echo ""
sleep 3

cd pythoncode
git clone https://github.com/DUNE-DAQ/drunc.git -b develop
cd ..

dunedaq_integtest_bundle.sh -r local --list

echo ""
echo -e "\U1F535 \U2705 Note that the drunc integtest *is now included* in the list of *local* tests which would be run. \U2705 \U1F535"
echo ""
echo ""
sleep 3

dunedaq_integtest_bundle.sh -r drunc

echo ""
echo -e "\U1F535 \U2705 Note that the drunc process_manager_test ran successfully using the bundle script. \U2705 \U1F535"
echo ""
echo ""

Type of change

• Optimization (non-breaking change that improves code/performance)

Testing checklist

• Full set of integration tests pass (dunedaq_integtest_bundle.sh)

Further checks

• Code is commented where needed, particularly in hard-to-understand areas